In the past two decades, curiosity has successfully attracted attention of numerous researchers in the field of Artificial Intelligence.
From the machine learning perspective, curiosity has been proposed as an algorithmic principle to focus learning on novel and learnable regularities, in contrast to irregular noise. For example, Schmidhuber [6] introduced curiosity into model-building control systems. In his work, curiosity is modeled as the prediction improvement between successive situations and is an intrinsic reward value guiding the selection of training examples such that the expected performance improvement is maximized. In autonomous robotic developmental systems, Oudeyer and Kaplan [7] proposed an Intelligent Adaptive Curiosity (IAC) mechanism and modeled curiosity as the prediction improvement between similar situations instead of successive situations.
Curiosity has also been modeled in exploratory agents to explore and learn in uncertain domains. For example, Scott and Markovitch [3] introduced curiosity for intelligent agents to learn unfamiliar domains. They adopted a heuristic that “what is needed is something that falls somewhere between novelty and familiarity”, where novelty is defined as a measure of how uncertain the agent is about the consequence of a stimulus. Uncertainty is implemented as Shannon's entropy of all the possible outcomes to a stimulus. The system can learn a good representation of the uncertain domain because it will not waste resources on commonly occurred cases but concentrate on less common ones. Another work is done by Macedo and Cardoso [1], who modeled curiosity in artificial perceptual agents to explore uncertain and unknown environments. This model relies on graph-based mental representations of objects and curiosity is implemented as the entropy of all parts that contain uncertainty in an object.
In creative agents, curiosity has been modeled as an intrinsic evaluation for novelty. For example, Saunders and Gero [8] developed a computational model of curiosity for “curious design agents” to search for novel designs and to guide design actions. A Self-Organizing Map (SOM) is employed as the “conceptual design space” for the agent. For a given input, novelty is implemented as a measure of cluster distance. This measure reflects the similarity between newly encountered design patterns with previously experienced ones. In Merrick and Maher's model [2], they utilized an improved SOM model named Habituated Self-Organizing Map (HSOM) to cluster similar tasks and novelty is calculated by a habituation function.
To summarize, in existing works, curiosity has been integrated into agents' learning modules and decision modules to enhance their performance. However, these agents can hardly be perceived to be believable by a human observer. There are two main reasons for this: (1) existing models lack a comprehensive psychological theory as background, and (2) agents perceive the environment on the machine language level (feature-based knowledge representation) rather than on the human language level (semantic knowledge representation).